Balancing Act: Constraining Disparate Impact in Sparse Models

Model pruning is a popular approach to enable the deployment of large deep learning models on edge devices with restricted computational or storage capacities. Although sparse models achieve performance comparable to that of their dense counterparts at the level of the entire dataset, they exhibit high accuracy drops for some data sub-groups. Existing methods to mitigate this disparate impact induced by pruning (i) rely on surrogate metrics that address the problem indirectly and have limited interpretability; or (ii) scale poorly with the number of protected sub-groups in terms of computational cost. We propose a constrained optimization approach that $\textit{directly addresses the disparate impact of pruning}$: our formulation bounds the accuracy change between the dense and sparse models, for each sub-group. This choice of constraints provides an interpretable success criterion to determine if a pruned model achieves acceptable disparity levels. Experimental results demonstrate that our technique scales reliably to problems involving large models and hundreds of protected sub-groups.Comment: Code available at https://github.com/merajhashemi/Balancing_Ac

GANGs: Generative Adversarial Network Games

Generative Adversarial Networks (GAN) have become one of the most successful frameworks for unsupervised generative modeling. As GANs are difficult to train much research has focused on this. However, very little of this research has directly exploited game-theoretic techniques. We introduce Generative Adversarial Network Games (GANGs), which explicitly model a finite zero-sum game between a generator ($G$) and classifier ($C$) that use mixed strategies. The size of these games precludes exact solution methods, therefore we define resource-bounded best responses (RBBRs), and a resource-bounded Nash Equilibrium (RB-NE) as a pair of mixed strategies such that neither $G$ or $C$ can find a better RBBR. The RB-NE solution concept is richer than the notion of `local Nash equilibria' in that it captures not only failures of escaping local optima of gradient descent, but applies to any approximate best response computations, including methods with random restarts. To validate our approach, we solve GANGs with the Parallel Nash Memory algorithm, which provably monotonically converges to an RB-NE. We compare our results to standard GAN setups, and demonstrate that our method deals well with typical GAN problems such as mode collapse, partial mode coverage and forgetting

A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale

Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad family of methods for training neural networks. It constructs a block-diagonal preconditioner where each block consists of a coarse Kronecker product approximation to full-matrix AdaGrad for each parameter of the neural network. In this work, we provide a complete description of the algorithm as well as the performance optimizations that our implementation leverages to train deep networks at-scale in PyTorch. Our implementation enables fast multi-GPU distributed data-parallel training by distributing the memory and computation associated with blocks of each parameter via PyTorch's DTensor data structure and performing an AllGather primitive on the computed search directions at each iteration. This major performance enhancement enables us to achieve at most a 10% performance reduction in per-step wall-clock time compared against standard diagonal-scaling-based adaptive gradient methods. We validate our implementation by performing an ablation study on training ImageNet ResNet50, demonstrating Shampoo's superiority over standard training recipes with minimal hyperparameter tuning.Comment: 38 pages, 8 figures, 5 table

Phylogenetic history demonstrates two different lineages of dengue type 1 virus in Colombia

Background: Dengue Fever is one of the most important viral re-emergent diseases affecting about 50 million people around the world especially in tropical and sub-tropical countries. In Colombia, the virus was first detected in the earliest 70′s when the disease became a major public health concern. Since then, all four serotypes of the virus have been reported. Although most of the huge outbreaks reported in this country have involved dengue virus serotype 1 (DENV-1), there are not studies about its origin, genetic diversity and distribution. Results: We used 224 bp corresponding to the carboxyl terminus of envelope (E) gene from 74 Colombian isolates in order to reconstruct phylogenetic relationships and to estimate time divergences. Analyzed DENV-1 Colombian isolates belonged to the formerly defined genotype V. Only one virus isolate was clasified in the genotype I, likely representing a sole introduction that did not spread. The oldest strains were closely related to those detected for the first time in America in 1977 from the Caribbean and were detected for two years until their disappearance about six years later. Around 1987, a split up generated 2 lineages that have been evolving separately, although not major aminoacid changes in the analyzed region were found. Conclusion: DENV-1 has been circulating since 1978 in Colombia. Yet, the phylogenetic relationships between strains isolated along the covered period of time suggests that viral strains detected in some years, although belonging to the same genotype V, have different recent origins corresponding to multiple re-introduction events of viral strains that were circulating in neighbor countries. Viral strains used in the present study did not form a monophyletic group, which is evidence of a polyphyletic origin. We report the rapid spread patterns and high evolution rate of the different DENV-1 lineages

Temporal changes in the epidemiology, management, and outcome from acute respiratory distress syndrome in European intensive care units: a comparison of two large cohorts

Background: Mortality rates for patients with ARDS remain high. We assessed temporal changes in the epidemiology and management of ARDS patients requiring invasive mechanical ventilation in European ICUs. We also investigated the association between ventilatory settings and outcome in these patients. Methods: This was a post hoc analysis of two cohorts of adult ICU patients admitted between May 1–15, 2002 (SOAP study, n = 3147), and May 8–18, 2012 (ICON audit, n = 4601 admitted to ICUs in the same 24 countries as the SOAP study). ARDS was defined retrospectively using the Berlin definitions. Values of tidal volume, PEEP, plateau pressure, and FiO2 corresponding to the most abnormal value of arterial PO2 were recorded prospectively every 24 h. In both studies, patients were followed for outcome until death, hospital discharge or for 60 days. Results: The frequency of ARDS requiring mechanical ventilation during the ICU stay was similar in SOAP and ICON (327[10.4%] vs. 494[10.7%], p = 0.793). The diagnosis of ARDS was established at a median of 3 (IQ: 1–7) days after admission in SOAP and 2 (1–6) days in ICON. Within 24 h of diagnosis, ARDS was mild in 244 (29.7%), moderate in 388 (47.3%), and severe in 189 (23.0%) patients. In patients with ARDS, tidal volumes were lower in the later (ICON) than in the earlier (SOAP) cohort. Plateau and driving pressures were also lower in ICON than in SOAP. ICU (134[41.1%] vs 179[36.9%]) and hospital (151[46.2%] vs 212[44.4%]) mortality rates in patients with ARDS were similar in SOAP and ICON. High plateau pressure (> 29 cmH2O) and driving pressure (> 14 cmH2O) on the first day of mechanical ventilation but not tidal volume (> 8 ml/kg predicted body weight [PBW]) were independently associated with a higher risk of in-hospital death. Conclusion: The frequency of and outcome from ARDS remained relatively stable between 2002 and 2012. Plateau pressure > 29 cmH2O and driving pressure > 14 cmH2O on the first day of mechanical ventilation but not tidal volume > 8 ml/kg PBW were independently associated with a higher risk of death. These data highlight the continued burden of ARDS and provide hypothesis-generating data for the design of future studies

The clinical relevance of oliguria in the critically ill patient : Analysis of a large observational database

Funding Information: Marc Leone reports receiving consulting fees from Amomed and Aguettant; lecture fees from MSD, Pfizer, Octapharma, 3 M, Aspen, Orion; travel support from LFB; and grant support from PHRC IR and his institution. JLV is the Editor-in-Chief of Critical Care. The other authors declare that they have no relevant financial interests. Publisher Copyright: © 2020 The Author(s). Copyright: Copyright 2020 Elsevier B.V., All rights reserved.Background: Urine output is widely used as one of the criteria for the diagnosis and staging of acute renal failure, but few studies have specifically assessed the role of oliguria as a marker of acute renal failure or outcomes in general intensive care unit (ICU) patients. Using a large multinational database, we therefore evaluated the occurrence of oliguria (defined as a urine output 16 years) patients in the ICON audit who had a urine output measurement on the day of admission were included. To investigate the association between oliguria and mortality, we used a multilevel analysis. Results: Of the 8292 patients included, 2050 (24.7%) were oliguric during the first 24 h of admission. Patients with oliguria on admission who had at least one additional 24-h urine output recorded during their ICU stay (n = 1349) were divided into three groups: transient - oliguria resolved within 48 h after the admission day (n = 390 [28.9%]), prolonged - oliguria resolved > 48 h after the admission day (n = 141 [10.5%]), and permanent - oliguria persisting for the whole ICU stay or again present at the end of the ICU stay (n = 818 [60.6%]). ICU and hospital mortality rates were higher in patients with oliguria than in those without, except for patients with transient oliguria who had significantly lower mortality rates than non-oliguric patients. In multilevel analysis, the need for RRT was associated with a significantly higher risk of death (OR = 1.51 [95% CI 1.19-1.91], p = 0.001), but the presence of oliguria on admission was not (OR = 1.14 [95% CI 0.97-1.34], p = 0.103). Conclusions: Oliguria is common in ICU patients and may have a relatively benign nature if only transient. The duration of oliguria and need for RRT are associated with worse outcome.publishersversionPeer reviewe

GAIT: A Geometric Approach to Information Theory = GAIT: Un Enfoque Geométrico a la Teoría de la Información

Abstract: We advocate the use of a notion of entropy that reflects the relative abundances of the symbols in an alphabet, as well as the similarities between them. This concept was originally introduced in theoretical ecology to study the diversity of ecosystems. Based on this notion of entropy, we introduce geometry-aware counterparts for several concepts and theorems in information theory. Notably, our proposed divergence exhibits performance on par with state-of-the-art methods based on the Wasserstein distance, but enjoys a closed-form expression that can be computed efficiently. We demonstrate the versatility of our method via experiments on a broad range of domains: training generative models, computing image barycenters, approximating empirical measures and counting modes. Resumen: En este artículo promovemos el uso de una noción de entropía que refleje las abundancias relativas de los elementos de un alfabeto, así como las similitudes entre ellos. Este concepto fue originalmente presentado en ecología teórica para estudiar la diversidad de un ecosistema. Con base en esta noción de entropía, nosotros analizamos diversos conceptos y teoremas en teoría de la información, y presentamos versiones de sensibles a la geometría de estos. Notablemente, nuestra propuesta de una divergencia estadística presenta un desempeño a la par con respecto a métodos de estado del arte basados en la distancia de Wasserstein, pero cuenta con una expresión analítica que puede ser calculada de manera eficiente. Demostramos la versatilidad de nuestro método a través de experimentos en un amplio rango de tareas: entrenamiento de modelos generativos, cálculo de baricentros entre imágenes, aproximación de medidas empíricas y el conteo de modas. Machine learning; Information theory; Geometry; Entropy; Generative models; Statistical divergence; Aprendizaje automático; Teoría de la información; Geometría; Entropía; Modelos generativos; Divergencia estadístic

Equivariant Mesh Attention Networks

Equivariance to symmetries has proven to be a powerful inductive bias in deep learning research. Recent works on mesh processing have concentrated on various kinds of natural symmetries, including translations, rotations, scaling, node permutations, and gauge transformations. To date, no existing architecture is equivariant to all of these transformations. Moreover, previous implementations have not always applied these symmetry transformations to the test dataset. This inhibits the ability to determine whether the model attains the claimed equivariance properties. In this paper, we present an attention-based architecture for mesh data that is provably equivariant to all transformations mentioned above. We carry out experiments on the FAUST and TOSCA datasets, and apply the mentioned symmetries to the test set only. Our results confirm that our proposed architecture is equivariant, and therefore robust, to these local/global transformations.Comment: Implementation can be found at: https://github.com/gallego-posada/ema

Altered Hypercoagulability Factors in Patients with Chronic Chagas Disease: Potential Biomarkers of Therapeutic Response.

Thromboembolic events were described in patients with Chagas disease without cardiomyopathy. We aim to confirm if there is a hypercoagulable state in these patients and to determine if there is an early normalization of hemostasis factors after antiparasitic treatment. Ninety-nine individuals from Chagas disease-endemic areas were classified in two groups: G1, with T.cruzi infection (n = 56); G2, healthy individuals (n = 43). Twenty-four hemostasis factors were measured at baseline. G1 patients treated with benznidazole were followed for 36 months, recording clinical parameters and performance of conventional serology, chemiluminescent enzyme-linked immunosorbent assay (trypomastigote-derived glycosylphosphatidylinositol-anchored mucins), quantitative polymerase chain reaction, and hemostasis tests every 6-month visits. Prothrombin fragment 1+2 (F1+2) and endogenous thrombin potential (ETP) were abnormally expressed in 77% and 50% of infected patients at baseline but returned to and remained at normal levels shortly after treatment in 76% and 96% of cases, respectively. Plasmin-antiplasmin complexes (PAP) were altered before treatment in 32% of G1 patients but normalized in 94% of cases several months after treatment. None of the patients with normal F1+2 values during follow-up had a positive qRT-PCR result, but 3/24 patients (13%) with normal ETP values did. In a percentage of chronic T. cruzi infected patients treated with benznidazole, altered coagulation markers returned into normal levels. F1+2, ETP and PAP could be useful markers for assessing sustained response to benznidazole

Variations in AT CL-ELISA levels.

<p>Months of follow-up predictive margins with 95% CI. Cutoff AT CL ELISA = 0.</p